Transforming Linear-threshold Learning Algorithms into Multi-class Linear Learning Algorithms

In this paper, we present a new type of multi-class learning algorithm called a linear-max algorithm. Linear-max algorithms learn with a special type of attribute called a sub-expert. A sub-expert is a vector attribute that has a value for each output class. The goal of the multi-class algorithm is to learn a linear function combining the sub-experts and to use this linear function to make correct class predictions. We will prove that, in the on-line mistakebounded model of learning, these multi-class learning algorithms have the same mistake bounds as a related two class linear-threshold algorithm. We will also show how sub-experts can be used to solve more traditional problems composed of real valued attributes. This leads to a natural extension of the algorithm to multi-class problems that contain both traditional attributes and sub-experts.